Repository navigation
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
triniwiz
force-pushed
the
feat/html-in-canvas-windows
branch
9 times, most recently
from
November 20, 2026 00:13
2cf499a to
aa81a81
Compare
triniwiz
marked this pull request as ready for review
November 20, 2026 00:17
triniwiz
force-pushed
the
feat/html-in-canvas-windows
branch
from
November 20, 2026 01:15
aa81a81 to
f7d20b8
Compare
HTML-in-Canvas on WinUI 3. The children of a content="drawable" canvas live in a transparent XAML Canvas on top of the SwapChainPanel, each in a Grid slot that carries the drawn geometry (a RenderTransform, a clip, and hit-test visibility), so XAML's own hit testing finds the view where it was drawn. XAML input follows neither a Projection nor a 4x4 TransformMatrix, so for a matrix with perspective the host sees each pointer message before XAML (it subclasses the island's windows): the view's slot gets the matrix's tangent at the point under the pointer, exact there, for mouse, pen and a touch's first contact alike. Hosted views keep the size they were given (no layout rounding), and snapshots are resampled to the requested pixel size, since RenderTargetBitmap takes whole DIPs. The host is native (NSCDrawableHost in canvas-napi): JS makes one update() call a frame. host/windows/xaml.rs declares the WinUI 3 members it calls, from the Windows App SDK 1.6 metadata (the windows crate covers only Windows.*). WinUI 3 has no per-element invalidation and only an async snapshot, so views are re-rendered with RenderTargetBitmap on a schedule that is hot for half a second after a change or input and backs off to once a second when idle, and compared (a quarter-size probe first). The async steps are polled from update(): XAML objects are not Send. captureElementImage and cropped uploads come from the last snapshot. A press on a hosted view no longer takes pointer capture and focus for the canvas. The htmlincanvas spec suite is registered on Windows. The demo follows the system theme.
triniwiz
force-pushed
the
feat/html-in-canvas-windows
branch
from
November 20, 2026 02:10
f7d20b8 to
92e8540
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HTML-in-Canvas on NativeScript Windows (WinUI 3).
Change
NSCDrawableHost(canvas-napi,host/windows/drawable.rs)Canvason top of theSwapChainPanel, each in its ownGridslot. Opacity hides them but keeps them hit-testable.RenderTransform(the only transform XAML input follows: it ignoresProjectionandUIElement.TransformMatrix);RectangleGeometryclip forcanvasClip;IsHitTestVisibleonly while the child has geometry.Raising uses
Canvas.ZIndex, so the element is never re-parented.picking.rs): the host subclasses the XAML island's windows and sees each pointer message before XAML. For every such view it maps the point back through the matrix and gives the slot the matrix's tangent there, which is exact at the pointer. XAML then hit-tests and delivers the event as usual, so mouse, pen and a touch's first contact all reach the view where it was drawn, as in Chrome.NSCDrawableHost.aim(x, y)does the same for a point without a pointer (the spec uses it).RenderToSizeAsynctakes whole DIPs.RenderTargetBitmap). Views are re-rendered and compared: a quarter-size probe first, then the full snapshot if it differs. The schedule is hot for 0.5 s after a change or input, then backs off from 66 ms to 1 s. It runs on time, not frames. Apaintevent follows a change by a frame or more.update()rather than given completion handlers, because windows-rs delegates must beSendand XAML objects are not. Input on the hosted views only sets an atomic flag.update()call a frame. Pixel comparison, BGRA loading, and the crop/scale forcaptureElementImageand cropped uploads (from the last snapshot) all happen in Rust.host/windows/xaml.rsdeclares the WinUI 3 members the host calls (thewindowscrate covers onlyWindows.*): each interface's IID and its vtable up to the last slot used, taken from the Windows App SDK 1.6.250205002 metadata (the version the NativeScript Windows runtime ships), plus the pointer/key event delegate. Published WinRT interfaces never change, and each is reached byQueryInterface.TS:
DrawableController.canvasCssSize(): XAML lays the panel out, so the Windows controller reads its size from there. A press on a hosted view no longer takes pointer capture and focus for the canvas. Thehtmlincanvassuite is registered on Windows; its hit-testing checks useVisualTreeHelper.FindElementsInHostCoordinates, which hit-tests as input does.Demo: follows the system theme (dark
.ns-darkcolors,DayNighton Android); thecanvas-htmlcard uses classes for it.Verified
On Windows 11 (Intel Iris Xe, 125% scale):
htmlincanvas: 25/25.canvas-htmldemo renders the rotated form. Real mouse clicks on the drawn button and typing into the drawn text field reach those views.htmlincanvas.hittesthas a perspective case at points 2 px inside and outside the button's edges, where an approximation taken at the button's center is wrong on both; it fails whenaimis skipped.The hit-testing checks call
VisualTreeHelper.FindElementsInHostCoordinates(Point, UIElement), which needs NativeScript/windows#55 to reach the Point overload. Loading canvas on Windows needs NativeScript/windows#54.Canvases that present through a
SwapChainPanelare not in the snapshots (RenderTargetBitmapdoes not capture them). A view whose hit geometry changes (an animation, or the pointer moving over a view with perspective) re-renders on every check, since XAML rasterizes through the transform and each one gives different pixels.